本指南将向您详细介绍如何使用 Hugging Face Transformers 和 TRL 针对自定义图片和文本数据集微调 Gemma，以执行视觉任务（生成商品说明）。您会了解到以下内容：
- 什么是量化低秩自适应 (QLoRA)
- 设置开发环境
- 创建和准备用于视觉任务的微调数据集
- 使用 TRL 和 SFTTrainer 微调 Gemma
- 测试模型推理，并根据图片和文本生成商品说明。
什么是量化低秩自适应 (QLoRA)
本指南演示了如何使用量化低秩自适应 (QLoRA)。由于它可以减少计算资源需求，同时保持高性能，因此已成为高效微调 LLM 的热门方法。在 QloRA 中，预训练模型会量化为 4 位，并且权重会被冻结。然后，附加可训练的适配器层 (LoRA)，并且仅训练适配器层。之后，适配器权重可以与基准模型合并，也可以保留为单独的适配器。
设置开发环境
第一步是安装 Hugging Face 库（包括 TRL）和数据集，以微调开放模型。
# Install Pytorch & other libraries
%pip install "torch>=2.4.0" tensorboard torchvision
# Install Gemma release branch from Hugging Face
%pip install "transformers>=4.51.3"
# Install Hugging Face libraries
%pip install --upgrade \
"datasets==3.3.2" \
"accelerate==1.4.0" \
"evaluate==0.4.3" \
"bitsandbytes==0.45.3" \
"trl==0.15.2" \
"peft==0.14.0" \
"pillow==11.1.0" \
protobuf \
sentencepiece
在开始训练之前，您必须确保已接受 Gemma 的使用条款。您可以在 Hugging Face 上接受许可，方法是点击模型页面（网址为 http://huggingface.co/google/gemma-3-4b-pt，或您所用的具有视觉功能的 Gemma 模型的相应模型页面）上的“同意并访问代码库”按钮。
接受许可后，您需要有效的 Hugging Face 令牌才能访问该模型。如果您是在 Google Colab 中运行，则可以使用 Colab Secret 安全地使用 Hugging Face 令牌；否则，您可以直接在 login 方法中设置令牌。由于您会在训练期间将模型推送到 Hub，因此请确保您的令牌也具有写入权限。
from google.colab import userdata
from huggingface_hub import login
# Login into Hugging Face Hub
hf_token = userdata.get('HF_TOKEN') # If you are running inside a Google Colab
login(hf_token)
创建和准备微调数据集
在微调 LLM 时，请务必了解您的应用场景和要解决的任务。这有助于您创建数据集来微调模型。如果您尚未定义用例，不妨回到起点重新思考。
例如，本指南重点介绍以下使用场景：
- 微调 Gemma 模型，为电子商务平台生成经过搜索引擎优化的简洁商品说明，专为移动搜索量身定制。
本指南使用 philschmid/amazon-product-descriptions-vlm 数据集，该数据集包含 Amazon 商品说明，包括商品图片和类别。
Hugging Face TRL 支持多模态对话。重要的部分是“image”角色，它会告知处理类应加载图片。结构应遵循以下格式：
{"messages": [{"role": "system", "content": [{"type": "text", "text":"You are..."}]}, {"role": "user", "content": [{"type": "text", "text": "..."}, {"type": "image"}]}, {"role": "assistant", "content": [{"type": "text", "text": "..."}]}]}
{"messages": [{"role": "system", "content": [{"type": "text", "text":"You are..."}]}, {"role": "user", "content": [{"type": "text", "text": "..."}, {"type": "image"}]}, {"role": "assistant", "content": [{"type": "text", "text": "..."}]}]}
{"messages": [{"role": "system", "content": [{"type": "text", "text":"You are..."}]}, {"role": "user", "content": [{"type": "text", "text": "..."}, {"type": "image"}]}, {"role": "assistant", "content": [{"type": "text", "text": "..."}]}]}
现在，您可以使用 Hugging Face Datasets 库加载数据集，并创建一个提示模板来组合图片、商品名称和类别，以及添加系统消息。数据集包含作为 Pil.Image 对象的图片。
from datasets import load_dataset
from PIL import Image
# System message for the assistant
system_message = "You are an expert product description writer for Amazon."
# User prompt that combines the user query and the schema
user_prompt = """Create a Short Product description based on the provided <PRODUCT> and <CATEGORY> and image.
Only return description. The description should be SEO optimized and for a better mobile search experience.
<PRODUCT>
{product}
</PRODUCT>
<CATEGORY>
{category}
</CATEGORY>
"""
# Convert dataset to OAI messages
def format_data(sample):
return {
"messages": [
{
"role": "system",
"content": [{"type": "text", "text": system_message}],
},
{
"role": "user",
"content": [
{
"type": "text",
"text": user_prompt.format(
product=sample["Product Name"],
category=sample["Category"],
),
},
{
"type": "image",
"image": sample["image"],
},
],
},
{
"role": "assistant",
"content": [{"type": "text", "text": sample["description"]}],
},
],
}
def process_vision_info(messages: list[dict]) -> list[Image.Image]:
image_inputs = []
# Iterate through each conversation
for msg in messages:
# Get content (ensure it's a list)
content = msg.get("content", [])
if not isinstance(content, list):
content = [content]
# Check each content element for images
for element in content:
if isinstance(element, dict) and (
"image" in element or element.get("type") == "image"
):
# Get the image and convert to RGB
if "image" in element:
image = element["image"]
else:
image = element
image_inputs.append(image.convert("RGB"))
return image_inputs
# Load dataset from the hub
dataset = load_dataset("philschmid/amazon-product-descriptions-vlm", split="train")
# Convert dataset to OAI messages
# need to use list comprehension to keep Pil.Image type, .mape convert image to bytes
dataset = [format_data(sample) for sample in dataset]
print(dataset[345]["messages"])
使用 TRL 和 SFTTrainer 微调 Gemma
现在，您可以对模型进行微调了。借助 Hugging Face TRL SFTTrainer，您可以轻松监督微调开放式 LLM。SFTTrainer 是 transformers 库中的 Trainer 的子类，支持所有相同的功能（包括日志记录、评估和检查点），但还添加了其他实用功能，包括：
- 数据集格式设置，包括对话格式和指令格式
- 仅根据完成情况进行训练，忽略提示
- 打包数据集以提高训练效率
- 支持参数高效微调 (PEFT)，包括 QloRA
- 准备模型和分词器以进行对话式微调（例如添加特殊标记）
以下代码会从 Hugging Face 加载 Gemma 模型和分词器，并初始化量化配置。
import torch
from transformers import AutoProcessor, AutoModelForImageTextToText, BitsAndBytesConfig
# Hugging Face model id
model_id = "google/gemma-3-4b-pt" # or `google/gemma-3-12b-pt`, `google/gemma-3-27-pt`
# Check if GPU benefits from bfloat16
if torch.cuda.get_device_capability()[0] < 8:
raise ValueError("GPU does not support bfloat16, please use a GPU that supports bfloat16.")
# Define model init arguments
model_kwargs = dict(
attn_implementation="eager", # Use "flash_attention_2" when running on Ampere or newer GPU
torch_dtype=torch.bfloat16, # What torch dtype to use, defaults to auto
device_map="auto", # Let torch decide how to load the model
)
# BitsAndBytesConfig int-4 config
model_kwargs["quantization_config"] = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=model_kwargs["torch_dtype"],
bnb_4bit_quant_storage=model_kwargs["torch_dtype"],
)
# Load model and tokenizer
model = AutoModelForImageTextToText.from_pretrained(model_id, **model_kwargs)
processor = AutoProcessor.from_pretrained("google/gemma-3-4b-it")
SFTTrainer 支持与 peft 的内置集成，这样您就可以轻松使用 QLoRA 高效地调优 LLM。您只需创建 LoraConfig 并将其提供给培训师即可。
from peft import LoraConfig
peft_config = LoraConfig(
lora_alpha=16,
lora_dropout=0.05,
r=16,
bias="none",
target_modules="all-linear",
task_type="CAUSAL_LM",
modules_to_save=[
"lm_head",
"embed_tokens",
],
)
在开始训练之前，您需要定义要在 SFTConfig 中使用的超参数，以及用于处理视觉处理的自定义 collate_fn。collate_fn 会将包含文本和图片的消息转换为模型可以理解的格式。
from trl import SFTConfig
args = SFTConfig(
output_dir="gemma-product-description", # directory to save and repository id
num_train_epochs=1, # number of training epochs
per_device_train_batch_size=1, # batch size per device during training
gradient_accumulation_steps=4, # number of steps before performing a backward/update pass
gradient_checkpointing=True, # use gradient checkpointing to save memory
optim="adamw_torch_fused", # use fused adamw optimizer
logging_steps=5, # log every 5 steps
save_strategy="epoch", # save checkpoint every epoch
learning_rate=2e-4, # learning rate, based on QLoRA paper
bf16=True, # use bfloat16 precision
max_grad_norm=0.3, # max gradient norm based on QLoRA paper
warmup_ratio=0.03, # warmup ratio based on QLoRA paper
lr_scheduler_type="constant", # use constant learning rate scheduler
push_to_hub=True, # push model to hub
report_to="tensorboard", # report metrics to tensorboard
gradient_checkpointing_kwargs={
"use_reentrant": False
}, # use reentrant checkpointing
dataset_text_field="", # need a dummy field for collator
dataset_kwargs={"skip_prepare_dataset": True}, # important for collator
)
args.remove_unused_columns = False # important for collator
# Create a data collator to encode text and image pairs
def collate_fn(examples):
texts = []
images = []
for example in examples:
image_inputs = process_vision_info(example["messages"])
text = processor.apply_chat_template(
example["messages"], add_generation_prompt=False, tokenize=False
)
texts.append(text.strip())
images.append(image_inputs)
# Tokenize the texts and process the images
batch = processor(text=texts, images=images, return_tensors="pt", padding=True)
# The labels are the input_ids, and we mask the padding tokens and image tokens in the loss computation
labels = batch["input_ids"].clone()
# Mask image tokens
image_token_id = [
processor.tokenizer.convert_tokens_to_ids(
processor.tokenizer.special_tokens_map["boi_token"]
)
]
# Mask tokens for not being used in the loss computation
labels[labels == processor.tokenizer.pad_token_id] = -100
labels[labels == image_token_id] = -100
labels[labels == 262144] = -100
batch["labels"] = labels
return batch
现在，您已拥有创建 SFTTrainer 所需的所有构建块，可以开始训练模型了。
from trl import SFTTrainer
trainer = SFTTrainer(
model=model,
args=args,
train_dataset=dataset,
peft_config=peft_config,
processing_class=processor,
data_collator=collate_fn,
)
通过调用 train() 方法开始训练。
# Start training, the model will be automatically saved to the Hub and the output directory
trainer.train()
# Save the final model again to the Hugging Face Hub
trainer.save_model()
在测试模型之前，请务必释放内存。
# free the memory again
del model
del trainer
torch.cuda.empty_cache()
使用 QLoRA 时，您只需训练适配器，而无需训练完整模型。这意味着，在训练期间保存模型时，您只会保存适配器权重，而不会保存完整模型。如果您想保存完整模型，以便更轻松地与 vLLM 或 TGI 等服务堆栈搭配使用，可以使用 merge_and_unload 方法将适配器权重合并到模型权重，然后使用 save_pretrained 方法保存模型。这会保存一个默认模型，该模型可用于推理。
from peft import PeftModel
# Load Model base model
model = AutoModelForImageTextToText.from_pretrained(model_id, low_cpu_mem_usage=True)
# Merge LoRA and base model and save
peft_model = PeftModel.from_pretrained(model, args.output_dir)
merged_model = peft_model.merge_and_unload()
merged_model.save_pretrained("merged_model", safe_serialization=True, max_shard_size="2GB")
processor = AutoProcessor.from_pretrained(args.output_dir)
processor.save_pretrained("merged_model")
测试模型推理并生成商品说明
训练完成后，您需要评估和测试模型。您可以从测试数据集中加载不同的样本，并针对这些样本评估模型。
import torch
# Load Model with PEFT adapter
model = AutoModelForImageTextToText.from_pretrained(
args.output_dir,
device_map="auto",
torch_dtype=torch.bfloat16,
attn_implementation="eager",
)
processor = AutoProcessor.from_pretrained(args.output_dir)
您可以通过提供商品名称、类别和图片来测试推理。sample 包含一个漫威动作人偶。
import requests
from PIL import Image
# Test sample with Product Name, Category and Image
sample = {
"product_name": "Hasbro Marvel Avengers-Serie Marvel Assemble Titan-Held, Iron Man, 30,5 cm Actionfigur",
"category": "Toys & Games | Toy Figures & Playsets | Action Figures",
"image": Image.open(requests.get("https://m.media-amazon.com/images/I/81+7Up7IWyL._AC_SY300_SX300_.jpg", stream=True).raw).convert("RGB")
}
def generate_description(sample, model, processor):
# Convert sample into messages and then apply the chat template
messages = [
{"role": "system", "content": [{"type": "text", "text": system_message}]},
{"role": "user", "content": [
{"type": "image","image": sample["image"]},
{"type": "text", "text": user_prompt.format(product=sample["product_name"], category=sample["category"])},
]},
]
text = processor.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
# Process the image and text
image_inputs = process_vision_info(messages)
# Tokenize the text and process the images
inputs = processor(
text=[text],
images=image_inputs,
padding=True,
return_tensors="pt",
)
# Move the inputs to the device
inputs = inputs.to(model.device)
# Generate the output
stop_token_ids = [processor.tokenizer.eos_token_id, processor.tokenizer.convert_tokens_to_ids("<end_of_turn>")]
generated_ids = model.generate(**inputs, max_new_tokens=256, top_p=1.0, do_sample=True, temperature=0.8, eos_token_id=stop_token_ids, disable_compile=True)
# Trim the generation and decode the output to text
generated_ids_trimmed = [out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
output_text = processor.batch_decode(
generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
return output_text[0]
# generate the description
description = generate_description(sample, model, processor)
print(description)
总结和后续步骤
本教程介绍了如何使用 TRL 和 QLoRA 对 Gemma 模型进行微调，以便执行视觉任务，尤其是生成商品说明。接下来，请参阅以下文档：